1. Field of the Invention
The present invention relates to a correcting method of lens distortion of an image.
2. Description of Related Art
An image taken by a video camera or a digital camera causes distortion (distortion aberration) depending upon a camera lens. The distortion aberration is generally extreme with a wide angle, increases toward a periphery of the image, and increases to 3% or more particularly with a zoom lens. Moreover, since the image is deformed in a so-called barrel or bobbin shape, this distortion aberration disadvantageously adversely affects estimation in computer vision, measurement using the image, and the like.
Parameters such as a distortion coefficient for determining a camera lens distortion are called a camera internal parameter, and cameral calibration means that the camera internal parameter is obtained. When the camera internal parameter can be obtained through camera calibration, the image lens distortion can be corrected by an image processing using the internal parameter.
A conventional camera calibrating method is disclosed in many research papers.
In the aforementioned conventional method, a relation between a certain point in a three-dimensional space and a point signal to the certain point (correspondence point) in the image is derived, and the internal parameter is estimated. There are two methods of obtaining the correspondence point as follows:
(1) A person designates the correspondence point with a mouse or the like.
(2) A grid intersection point, a polygon or polyhedron apex, or another characteristic point is automatically detected.
Since either one of these methods includes a measurement error, a large number of correspondence points have to be given in order to inhibit an error influence. That is, the more the number of correspondence points is, the more precise the obtained internal parameter becomes. Therefore, correspondence of an enormous number of (several tens to several hundreds) points has to be designated.
Moreover, these methods have the following problems.
In the method (1) in which the person designates the correspondence point with the mouse or the like, the error is large, operation is monotonous, and much labor is necessary. Furthermore, fatigue increases the error during designation.
In the method (2) (characteristic point detecting method) of automatically detecting the characteristic point by the image processing, the number of points detectable by the processing is limited. For example, when the grid intersection point is detected, the number of intersection points is only 88 in total even with 8 longitudinal lines and 11 lateral lines. It is said that at least 200 points or more are necessary for obtaining the parameter with good precision, and it is necessary to prepare the corresponding grid, polyhedron, and the like.
The present invention has been developed to solve the aforementioned various problems. That is, an object of the present invention is to provide an image lens distortion correcting method in which (1) no correspondence point needs to be given, (2) any special tool or object is not used, and (3) all points on an image are used, so that an internal parameter can automatically be obtained with high precision, and an image lens distortion can be corrected with high precision.
According to the present invention, there is provided an image lens distortion correcting method comprising: an image printing step (A) for printing an arbitrary image I1 in a computer; an image pickup step (B) for taking the printed image I1 with a camera having a lens distortion and obtaining a photographed image I2 in the computer; a parameter estimating step (C) for obtaining a parameter xcex8 such that the image I1 is artificially distorted with the parameter xcex8 and an obtained distorted image I1ud agrees with the photographed image I2; and an image correcting step (D) for using the obtained parameter xcex8 to correct the image taken by the camera.
According to the aforementioned method of the present invention, first the arbitrary image I1 (Computer graphics, a picture extracted with a scanner, or an image taken by a digital camera) is prepared as a calibration pattern in the computer. Subsequently, the image I1 is printed with a printer having no distortion. The printed image I1 is taken by a camera to be corrected, and the distorted photographed image I2 is obtained. When the certain internal parameter xcex8 is given, the image I1 can artificially be distorted. When the artificially distorted image I1ud is compared with the distorted image I2 and the images accurately equal with each other, the given parameter xcex8 is desirable, and the obtained parameter xcex8 can be used to correct the camera lens distortion.
According to a preferred embodiment of the present invention, the parameter xcex8 includes a position correction parameter xcex8u for correcting a position, and a distortion correction parameter xcex8d for correcting the distortion.
The position correction parameter xcex8u is a parameter for conversion to an image I1u with the corrected position from the image I1. The conversion parameter is obtained in a least-squares method such that a difference r=I1(p)xe2x88x92I1u(p+u) between a luminance value I1(p) of a point p in the image I1 and a luminance value I1u(p+u) of a point p+u in the image I1u corresponding to the point p is minimized entirely on the image. The obtained parameter is used as the position correction parameter xcex8u.
Moreover, the distortion correction parameter xcex8d is a parameter for conversion to the image Iiu from the distorted image I1ud. The conversion parameter is obtained in the least-squares method such that a difference r=I2(p)xe2x88x92I1u(f(p)) between a luminance value I2(p) of the point p in the image I2 and a luminance value I1u(f(p)) of a point f(p) in the image I1u corresponding to the point p is minimized entirely on the image. The obtained parameter is used as the distortion correction parameter xcex8d.
Furthermore, in the image correcting step (D), the obtained distortion correction parameter xcex8d is used to correct the image taken by the camera.
That is, as shown in FIG. 1, a position in the photographed image I2 in which the original image I1 is projected is not known. Therefore, two stages are performed in order to obtain the distorted image I1ud. That is, first the position corrected image I1u is obtained from the image I1, subsequently the image I1u is distorted and the distorted image I1ud is obtained.
First Stage
First, the parameter xcex8u for conversion to the image I1u from the image I1 is obtained. This does not include any internal parameter. The point in I1u corresponding to the point p in the image I1 deviates by u and is represented by p+u. Here, u changes with p and xcex8u. When the image I1 exactly equals with the photographed image I2, the luminance value I1 (p) in p has to be equal to the luminance value I2(p+u) in p+u. That is, the luminance difference r=I1(p)xe2x88x92I2(p+u) in each point has to be 0. An evaluation function xcexa3r2 is obtained by summing squared differences for all the points, and the parameter xcex8u is automatically obtained by repeating calculations such that the evaluation function is minimized.
Second Stage
Subsequently, the parameter xcex8d for conversion to the image I1u from the distorted image I1ud is obtained. This is the internal parameter such as a distortion coefficient. This stage is similar to the first stage, but instead of the conversion to the distorted image I1ud from the image I1u, a reverse conversion to the image I1u from the distorted image I1ud is considered.
The point in image I1u corresponding to the point p in the photographed image I2 is represented by f(p). Here, f( ) changes with p and xcex8d.
When the photographed image I2 exactly equals with the image I1u, the luminance value I2(p) in p has to be equal to the luminance value I1u(f(p)) in f(p). That is, the luminance difference r=I2(p)xe2x88x92I1u(f(p)) in each point has to be 0. The evaluation function xcexa3r2 is obtained by summing squared differences for all the points, and the parameter xcex8u is automatically obtained by repeating calculations such that the evaluation function is minimized.
Correction
The internal parameter xcex8d used for obtaining the distorted image I1ud from the image I1 can be used to conversely obtain I1 from I1ud. That is, a processing for correcting the applied distortion can be performed.
When the image I2 taken by the camera is subjected to the same processing as described above, the image with the corrected distortion can be obtained.
In the parameter estimating step (C), it is preferable to alternately repeatedly use the position correction parameter xcex8u and the distortion correction parameter xcex8d and obtain the parameter xcex8.
Moreover, the parameter xcex8 is a parameter for conversion to the image I1ud from the image I1. The conversion parameter is obtained in the least-squares method such that a difference r=I1(p)xe2x88x92I2(d(p+u(p)) between a luminance value I1(p) of the point p in the image I1 and a luminance value I2(d(p+u(p)) of a point d(p+u(p)) in the image I2 corresponding to the point p is minimized entirely on the image. The obtained parameter is used as the parameter xcex8.
According to the method, the parameter xcex8 can be estimated by a single step.
Other objects and advantageous characteristics of the present invention will be apparent from the following description with reference to the accompanying drawings.